service-automation: populate the contained-failure visibility contract — FlowRunSummary.failed fold, loop iteration through try_catch, $error.iteration / $error.item, failed= on the summary line - #15609

📓 Docs Drift Check

9 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run.

# while this PR is open — GitHub drops the merge commit once it closes
git fetch origin 026f57374ae4e1cf1c75d8961b66efecbd00489c && git checkout 026f57374ae4e1cf1c75d8961b66efecbd00489c
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin ba426b0f091b8d5901acf10bb8e649b514349f49 cb2bc181bee85fd78f87498695c7c3d523072d2e && git checkout -B drift-repro ba426b0f091b8d5901acf10bb8e649b514349f49 && git merge --no-ff cb2bc181bee85fd78f87498695c7c3d523072d2e

node scripts/docs-audit/affected-docs.mjs --json ba426b0f091b8d5901acf10bb8e649b514349f49

Contract review (Clause-②) — PR #15609 @ 56eeff22e

FUSE: serving model claude-fable-5-1; CONTRACT_REVIEW_TIER = 'claude-fable-5-1' (scripts/pm/dispatch-gates.mjs:8659) — exact match, review valid.

VERDICT: CHANGES REQUIRED — the engine work passes every attack below; what is owed is docs-only (one stale literal line this PR's own pin contradicts, one over-broad reading of failed=0) plus two issues to file for defects found one layer past the diff. No code change is required.

Reviewed in a dedicated worktree at 56eeff22e (git status 0 lines before and after every step; pnpm install exit 0). Three-dot origin/main...HEAD: 9 files / +684 / −19, as the PM measured. Clean-tree gates, exit codes by redirect: package suite 107 files / 1290 tests, exit 0; typecheck exit 0; check-engine-double-contract.mjs --self-test exit 0 and the real run exit 0; require('dist/index.cjs') exit 0 and import('dist/index.js') exit 0 after a real pnpm --filter …service-automation... build (exit 0).

Acceptance reproduction — confirmed

My own flow, not the PR's fixture, spelled exactly as the card: loop { body: [ try_catch { try: [notify], catch: [assignment] } ] } with nothing else in catch ($error read from the enclosing scope by a node after the loop — the binding lives in the shared Map). Real AutomationEngine, 5 rows, third ownerless:

• run completed; loop-body iterations [0,1,2,3,4]; summary line status=completed selected=5 acted=4 skipped=0 failed=1
• the catch region's assignment step: { parentNodeId:'guard', regionKind:'catch', iteration:2 }; the failing notify step: { regionKind:'try', iteration:2 }
• $error = {"nodeId":"guard","message":"Node 'notify' failed: …(c3)","iteration":2,"item":{"id":"c3","owner":null}}, TryCatchErrorValueSchema.safeParse ok
• two-caught variant → failed=2.

Item 1 — the unmeasured convention — confirmed

Hunted every path where an absent count could become 0:

• Zod: execution.zod.ts:263 is .optional() with no .default (read, not grepped).
• Producer: summarizeRun always emits (run-summary.ts:153); the line guard is !== undefined (run-summary.ts:206), so failed=0 prints and absent prints nothing.
• Persistence, both branches of serializeSummaryBounded (suspended-run-store.ts:849-862): the full branch is JSON.stringify(summary); the compacted branch spreads failed: summary.failed, and undefined stringifies away.
• Rehydration: parseJson(row.summary_json, undefined) at suspended-run-store.ts:589 is a raw JSON.parse, no merge. Driven through the real listHistory read path with three rows: older (no key) → 'failed' in summary === false, line …skipped=0 with no token; failed:0 → …skipped=0 failed=0; compacted older (detailOmitted:true, no key) → absent.
• Log meta engine.ts:6581 mirrors the unmeasured line above it; the emit site only ever logs a freshly folded summary.
• grep -rn 'failed ??\|failed ||' over the package's non-test src: zero hits for a summary (notify-node.ts:417 failed: result.failed is the unrelated hit that proves the grep ran).

Item 2 — the fold and its stated bound — confirmed against the declared formula; spec inconsistency to file (see N2)

failed += node.failures in the loop that settles node.status (run-summary.ts:135-145), one accumulator over the published array; my acceptance run asserts summary.failed === Σ nodes[].failures and guard: {runs:5, failures:0, status:'success'} / notify: {runs:5, failures:1}. The subflow bound the PR states is real and I measured it: parent loop { subflow(child) }, child containing its own failure → parent line status=completed selected=5 acted=4 skipped=0 failed=0 while the five child summaries carry failed=[0,0,0,0,1]. acted rolled up, failed did not. That follows from the declaration: subflow-node.ts (blob 8e069f46, identical to main) rolls up only selected/acted/unmeasuredEffect through ExecutionStepMetrics, which declares no failure slot, so Σ nodes[].failures cannot see a child's contained failure without a spec change. When the child fails rather than contains, the parent's subflow node failure is counted (call: {runs:5, failures:1}, parent failed=1) — consistent. Not a defect in this PR; the declaration is what disagrees with itself (execution.zod.ts:218-222 says a subflow rolls its child's totals up and the summary answers "what did this run cause"; :251-264 says failed is the node fold).

Item 3 — call-site forwarding, #14414 fence — confirmed

• Byte-unchanged by blob id: parallel-node.ts 68c475075c… on both origin/main and HEAD. engine.ts differs (c5f075e2… → d765ab2e…) by exactly 4 inserted lines at 6575-6581 (log meta); runRegion's tagger at engine.ts:7848-7862 is not in the diff.
• Branch index preserved under the new tagging: try_catch inside a parallel branch inside a loop → branch steps ga,0 noop,1 ×5 (regionKind:'parallel-branch'), while the try/catch region's own steps carry the loop index (notify,try,2,failure / capA,catch,2) and $error.iteration===2.
• The fence pin does discriminate: mutating the tagger to also write iteration onto already-tagged steps (} else if (grouping.iteration !== undefined) { step.iteration = grouping.iteration; }, injected=1 on disk) turned exactly leaves 'parallel' branch tagging exactly as it was red — because it asserts every(iteration===0) across 5 loop rows, a loop-index bleed shows as 1..4.

Item 4 — loop-frame.ts / AsyncLocalStorage — confirmed

Scope identity is sound because engine.execute always allocates new Map() (engine.ts:8465; resume rebuilds from persisted at :5298), so a child's scope is never === its parent's. Leak attempts, all on the real engine:

• subflow child with its own loop, invoked from a parent loop → child binds its own iteration:1 / item:k1 on all five calls, never the parent's 0..4;
• map child inside a parent loop → $error carries neither key;
• nested loops → inner wins ([2,'c3'] for both outer groups);
• two loops in two parallel branches, bodies interleaved with 3/5 ms delays, every row failing → 5 catches, each $error consistent with its own branch and row.
  Removing the scope check (return frame;) turns the subflow-leak pin red. runRegion refuses durable pause inside a region (engine.ts:7869-7872), so no resume path re-enters a body without its frame. node:async_hooks is present in both dist entries (cjs 2 / esm 1 hits) and both entries load; the package already ships node:crypto (http-nodes.ts:6), is node-only (engines.node >=22, no browser field), so the builtin is not a new class of dependency.

Item 5 — $error binding and the code widening — confirmed

Outside a loop: neither key (dev pin; mutation M2 below proves the pin distinguishes absent from key-present-undefined). Inside a loop with a coded failure: {"nodeId":"g","message":…,"code":"DUPLICATE_RECORD","iteration":2,"item":{…}}, safeParse ok — and parsed.data keys are nodeId, message, iteration, item: a consumer normalising through the schema loses code, which is exactly #14954's content. The widening is a single local annotation at the binding site (try-catch-node.ts:235) with a comment naming #14954 — one-line removal when the schema gains code, not a second contract.

Item 6 — the double-contract red — confirmed

Ledger blob scripts/engine-double-contract.baseline.json = f4bfac2a3… on both origin/main and HEAD; gate self-test and real run both exit 0 with the ledger untouched; the gate's per-file output shows run-summary.test.ts pinned [delete] / pinned [findOne]. I did not re-derive the "5"; the literal const engine: any = { count in that file is 3 on main and 3 on HEAD.

Item 7 — the four green pins — confirmed, each discriminates

Anything new

N1 — defect, this PR (docs): content/docs/automation/flows.mdx:875 prints the run summary line as literal log output: …selected=30 acted=0 skipped=30 gate=check_stalled->send_nudge:30. After this PR that run prints …skipped=30 failed=0 gate=… — this PR updated its own pin to exactly that (run-summary.test.ts:150-156) and left the doc's copy of the same line stale, directly above the table row that documents failed and says "Absent on a run that did not track it". content/docs/automation/ is a hand-written tree (AGENTS.md doc guardrails: only references/ and releases/ are off-limits in a code PR), and the docs-drift bot on this PR says itself it cannot see this page.

N2 — spec-lane inconsistency, not this PR's code: per item 2, failed=0 on a parent that delegates rows to a subflow reads "nothing failed" (the changeset's words) while the children lost rows. The two spec paragraphs conflict for that case; the resolution (a failure slot on ExecutionStepMetrics that subflow/map roll up, or an explicit carve-out on failed) is a domain:spec decision. Third in the series after #14954 / #15606.

N3 — pre-existing engine defect, out of this PR's surface (file it): a map node inside a loop body runs its collection once, on the first iteration only. map-node.ts keeps ${node.id}.$mapState in the shared scope and never deletes it after "All items done" (variables.set(stateKey, state) ~line 212; zero variables.delete(stateKey) hits, control: the sibling $mapItemDone is deleted at lines 132-133), so iterations 2..n see started === collection.length and run nothing while the step reports success. Measured: 5 iterations × 2 items → 2 child runs, map step success on all five, run completed, failed=0. Silent partial work is the class #14456 exists to expose, and this one is invisible even to the new counter.

N4 — observation, #14414's: under a parallel branch inside a loop, the try/catch region's steps now carry the loop index while their container's own step carries the branch index; regionKind disambiguates. Unchanged writes by parallel, but it is the overload #14414 owns.

N5 — observation: $error.item is collection[i] captured at publish time; if the try region reassigns the iterator variable before failing, item still names the original row (measured: [2,'c3'] after an assignment overwrote currentCase). Arguably the better answer; the spec text says "the iteratorVariable value" — a one-word clarification at most.

Changes required (no code)

1. content/docs/automation/flows.mdx:875 — bring the literal line to what the run now prints: … skipped=30 failed=0 gate=check_stalled->send_nudge:30, matching run-summary.test.ts:150-156.
2. Narrow the "failed=0 is the reading 'nothing failed'" sentence (changeset, run-summary.ts:197-205 comment, and the failed row at flows.mdx) to what item 2 measured: no node execution of this run failed — a subflow child's contained failures stay on the child's row. Cite the N2 issue.
3. File N2 (domain:spec) and N3 (domain:services, map-in-loop) as issues and reference them from the PR body's "Out of scope" list, alongside spec: TryCatchErrorValueSchema silently strips the code key the engine now binds — the declared "ONE shape" and the runtime shape have diverged (#14419 follow-up) #14954 / [finding] service-automation: FlowRunSummary.failed has no queryable column on sys_automation_run, so "which runs lost rows?" is a summary_json scan #15606.

Generated by Claude Code

Contract review (Clause-②) — round 2 — PR #15609 @ cb2bc181b

FUSE: serving model claude-fable-5-1; CONTRACT_REVIEW_TIER = 'claude-fable-5-1' (scripts/pm/dispatch-gates.mjs:8659) — exact match, review valid.

VERDICT: PASS. The three docs-only items from round 1 (comment 5547800535) are done, the release-notes call is right, and the PR is still a draft. One new thing was found (four more historical copies of the summary line, in changesets-generated CHANGELOGs); the disposition I chose is PASS with it noted — it makes no published statement false and the same "do not touch" judgement covers it.

Reviewed in a dedicated worktree at cb2bc181b (git status 0 lines). The round-2 delta 56eeff22e..cb2bc181b is one commit, 3 files, +20/−7, as the PM measured. run-summary.ts: every +/− line stripped by a comment-prefix filter leaves 0 lines (grep exit 1; control: the same filter over the .mdx diff leaves 4). run-summary.test.ts is byte-identical between the two heads (git diff --quiet exit 0), so the pin compared below is the one round 1 measured green — nothing executable changed and I re-ran nothing, deliberately.

Item 1 — flows.mdx:875 vs the pin — confirmed

The pin literal (run-summary.test.ts:151-157, two concatenated strings) reconstructed and compared to the doc line token by token: same 11 token keys in the same order, and the span selected=30 acted=0 skipped=30 failed=0 gate= is byte-equal in both. The three tokens that differ — run=run_a1b2/run_1, durationMs=142/42, gate=check_stalled->send_nudge:30/gate->nudge:30 — are the doc's illustrative fixture values, present unchanged in the merge-base copy (git show 900334a56:content/docs/automation/flows.mdx, line 875), so what round 1 asked for (… skipped=30 failed=0 gate=…) is exactly what moved.

Item 2 — the narrowed failed=0 reading, three places, each citing #15617 — confirmed

• content/docs/automation/flows.mdx:884 — the failed row: "failed=0 therefore reads 'no node execution of this run failed' … a subflow child's own contained failures are counted on the CHILD's summary, not folded up here the way acted is ([spec: FlowRunSummary's two paragraphs disagree for a subflow parent — failed is declared a node fold, while the summary is declared to answer "what did this run cause" and roll a child's totals up #15617])".
• packages/services/service-automation/src/run-summary.ts:205-214 — formatRunSummaryLine's comment: "NO NODE EXECUTION OF THIS RUN FAILED … a child run that CONTAINED failures of its own reports them on the child's summary and the parent still prints failed=0 — measured, alongside the control where a child that FAILS rather than contains does reach the parent's count … reconciled in spec: FlowRunSummary's two paragraphs disagree for a subflow parent — failed is declared a node fold, while the summary is declared to answer "what did this run cause" and roll a child's totals up #15617".
• .changeset/contained-failure-visibility.md:12 — "Read failed=0 precisely: no node execution of this run failed … see spec: FlowRunSummary's two paragraphs disagree for a subflow parent — failed is declared a node fold, while the summary is declared to answer "what did this run cause" and roll a child's totals up #15617".

The sentence is true as round 1 measured it (item 2 of 5547800535: parent loop { subflow(child) } → parent failed=0, children failed=[0,0,0,0,1]; a child that fails rather than contains → call: {runs:5, failures:1}, parent failed=1). The new comment states both the measurement and the control exactly as measured; not re-derived here. #15617 exists and is open; its title is the two-paragraphs disagreement the prose describes.

Item 3 — "Out of scope" names #15617 and #15616 beside #14954 / #15606 — confirmed

PR body, section "Out of scope, not widened into": four bullets, each saying what the issue is and why it is not addressed here. Cross-checked against the live issues: all four state: open; #15617 = the subflow parent declaration conflict (matches the bullet), #15616 = map-in-loop runs its collection once and completes green (matches, including the "silent partial work the counter cannot see" framing), #14954 = the code key the schema strips, #15606 = no failed column on sys_automation_run.

Release notes — the right call, and stated, not silent — confirmed

git diff --name-only 900334a56 cb2bc181b -- content/docs/releases/ → 0 (control: -- content/docs/automation/ → 1, flows.mdx); git diff --quiet … -- content/docs/releases/v17.mdx exit 0. AGENTS.md:730 makes content/docs/releases/ RELEASE-OWNED: "Never edit in a code PR … Factual error on a releases page → dedicated docs-only PR or an issue, never a rider on code changes." v17.mdx:1607 is not even a factual error — it records what v17.0 printed, and failed= did not exist then. The omission is stated in the PR body (section "Review round 2") and in the commit message of cb2bc181b (third paragraph). Right call.

The two refusals to overclaim — confirmed

• No ablation this round. The PR body's only ablation section is round 1's engine ablation; the "Review round 2" paragraph claims gates / suite / typecheck / lint / ADR-0087 re-runs and no ablation. Where the dev "said so plainly" was the dev report, which I cannot see — NOT MEASURED as to phrasing; what is measured is that the PR does not overclaim one, and an ablation of a comment-only diff could not discriminate anyway.
• Still a draft. pull_request_read get → "draft": true, state: open, mergeable_state: blocked, updated_at 2026-09-05T00:12:25Z (after the round-2 push).

Anything new

N1 — noted, PASS. A wider grep (acted=[0-9]+ skipped=[0-9]+, whole repo minus node_modules/dist/.git) finds four more copies of the stale line beyond v17.mdx:1607: packages/spec/CHANGELOG.md:27034, :66515, packages/services/service-automation/CHANGELOG.md:3328, :9607. These are changesets-generated (.changeset/config.json → @changesets/cli/changelog; the service-automation copy sits under ## 17.0.0) — release records of the same class as v17.mdx, untouched by this PR (0 CHANGELOG.md in the diff). The dev's "exactly one more" is a docs-tree count; neither the PR body nor the commit message claims exclusivity (both say v17.mdx "carries the same line"), so nothing published is false, and the judgement is identical for them.

N2 — observation only. The TS comment writes Sigma nodes[].failures where the changeset writes Σ — presumably to keep the source ASCII; harmless.

Nothing to change. Landing is the PM's; not merged, not marked ready, branch untouched.

Generated by Claude Code